SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model. To address the above issue in a more principled manner, we propose a new computational framework for robust and efficient fine-tuning for pre-trained language models. Specifically, our proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the capacity of the model; 2. Bregman proximal point optimization, which is a class of trust-region methods and can prevent knowledge forgetting. Our experiments demonstrate that our proposed method achieves the state-of-the-art performance on multiple NLP benchmarks.Comment: The 58th annual meeting of the Association for Computational Linguistics (ACL 2020

Modelling and control of an elastically joint-actuated cart-pole underactuated system

This paper investigates the modelling and closedloop tracking control issues of a novel elastic underactuated multibody system. A torsional inverted pendulum cart-pole system with a single rotary actuator at the pivot of the cart is proposed. The system dynamics which incorporates with motion planning is firstly described. An optimization procedure is then discussed to plan the feasible trajectories that not just meet the performance requirements but also obtain optimality with respect to the cart displacement and average velocity. A closed-loop tracking controller is designed under collocated partial feedback linearization (CPFL). Subsequent presentation of simulation demonstrates that the proposed system is promising as compared to the previous work. The paper concludes with the application of our novel scheme to the design and control of autonomous robot systems

Modelling, dynamic analysis and control of capsubot systems with stable propulsion for medical and recovery assistances

The growth of medical robots since the mid-1980s has been striking. From a few initial efforts in stereotactic brain surgery, orthopaedics, endoscopic surgery, microsurgery, and other areas, the field has expanded to include commercially marketed, clinically deployed systems, and a robust and exponentially expanding research community. Obscure gastrointestinal (GI) bleeding, Crohn disease, Celiac disease, small bower tumors, and other disorders that occur in the GI tract have always been challenging to be diagnosed and treated due to the inevitable difficulty in accessing such a complex environment within the human body. Robot-assisted minimally invasive surgery has become an choice

Geometric techniques for trajectory planning and chaos control of a bio-inspired autogenetic capsule robot

Biological systems achieve energy efficient and adaptive behaviours through extensive internal and external compliance interactions. Active dynamic compliance are created and enhanced from musculoskeletal system (joint-space) to external environment (task-space) amongst the underactuated motions. The terminology bio-inspiration implies the understanding of fundamental principles underlying the motion behaviours of animals and humans and transfers these principles into the development of robotic systems. For example, during walking, the muscles constantly change their stiffness and damping when the leg is swinging forward and the foot is put on the ground. This idea enables the exploration in robotic systems with flexible elements—viscoelasticity to mimic the compliant motion of biological muscles. Underactuated systems with viscoelastic actuation are similar to these biological systems, in that their self-organisation and overall tasks must be achieved by coordinating the subsystems and dynamically interacting with the environment

On periodically pendulum-diven systems for underactuated locomotion: a viscoelastic jointed model

This paper investigates the locomotion principles and nonlinear dynamics of the periodically pendulum-driven (PD) systems using the case of a 2-DOF viscoelastic jointed model. As a mechanical system with underactuation degree one, the proposed system has strongly coupled nonlinearities and can be utilized as a potential benchmark for studying complicated PD systems. By mathematical modeling and non-dimensionalization of the physical system, an insight is obtained to the global system dynamics. The proposed 2-DOF viscoelastic jointed model establishes a commendable interconnection between the system dynamics and the periodically actuated force. Subsequently, the periodic locomotion principles of the actuated subsystem are elaborately studied and synthesized with the characteristic of viscoelastic element. Then the analysis of qualitative changes is conducted respectively under the varying excitation amplitude and frequency. Simulation results validate the efficiency and performance of the proposed system comparing with the conventional system